Advanced Usability Testing Methodology

Most usability professionals learned their skills by reading books on usability testing or by watching someone else prepare and conduct tests. But how do we know that the particular way we learned to do usability testing is still the best way? Until the Comparative Usability Evaluation (CUE) project, there was little opportunity for usability professionals to objectively compare their various approaches. As it turns out, there are some striking differences in effectiveness. This tutorial gives rare insights into the practical doings of usability professionals, normally shrouded behind walls of confidentiality. This tutorial is not based on the personal opinions of one or two instructors but on the practical accumulated experience both good and bad of the eleven professional usability labs who participated in the CUE-1 and CUE-2 projects, conducting controlled usability tests in realistic, industrial settings. In this tutorial participants will learn: • What constitutes quality in usability testing. • Options available in planning a usability test, and how the choices may affect the outcome. • What makes a good usability test scenario, and how test scenarios impact the quality of a usability study. • What a usability problem is, and how usability problems are identified. • How many test participants are required to find for example 85% of the usability problems in a product. • Required characteristics of a usability report to assure its communicative value. • Novel communication techniques that are vastly superior to traditional usability problem communication through paper reports and video tapes. Insight from the professional lab studies will enable participants to assess and improve their abilities in usability test planning, scenario design, usability reporting and usability problem communication.